CONTROL DEVICE HAVING INTEGRATED 
MOUSE AND REMOTE CONTROL CAPABILITIES 



BACKGROUND 

5 This invention relates generally to wireless control devices and, more particularly, 

relates to a wireless control device having integrated mouse and remote control 
capabilities. 

Wireless mice are well known in the art as a means for providing cursor 
manipulation and selection input to personal computers. However, as personal computer 

10 ("PC") functionality expands into home entertainment applications, such as exemplified 
by PCs using Microsoft's Windows Media Center Edition software, a need has also 
developed for the provision of remote control functionality in conjunction with personal 
computers. Figure 1 illustrates such a Media Center Edition PC product 100 (prior art) 
which is supplied with both a mouse 102 and a remote control 104. Although not 

15 illustrated in Figure 1, it is anticipated that such PCs will be interfaced to home 

entertainment appliances such as televisions, satellite set top boxes, etc., to eventually 
become an integral part of an overall home entertainment system. In such cases universal 
remote control functionality, i.e. the ability to control multiple consumer entertainment 
appliances of different types and different manufactures, will also be a desirable feature 

20 in a remote control supplied with such a PC. 

As a result of this convergence of PC and home entertainment equipment a need 
thus exists for a hand-held user input device which combines the control functionalities 
required for both PCs and home entertainment appliances. Unfortunately, the 
requirements for each of these classes of device are different. Control of a PC is best 
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effected via a "mouse" used to manipulate a cursor and make selections. Mouse- 
generated x-y positioning input is, however, not easily adapted to operation of 
conventional entertainment equipment, for which a button-oriented remote control is 
better suited. Furthermore, many wireless mice operate using RF transmission to avoid 
5 line-of-sight issues when operated from a cluttered surface, in contrast to most 

entertainment equipment remote controls which are intended to be picked up and pointed 
at the controlled apparatus, and thus generally use IR as a transmission medium. 

Accordingly, a need exists for a control device having integrated mouse and 
remote control capabilities which thus combines the control functionalities required for 
10 both PCs and home entertainment appliances. 

SUMMARY 

In accordance with these needs, the subject invention is directed to a wireless 
control device having integrated mouse and remote control capabilities. In an exemplary 

15 embodiment, the wireless mouse functionality is implemented using RF transmissions, 
while the remote control functionality can be implemented using RF and/or IR 
transmissions from the control device. The control device is adapted to detect and 
configure the mouse and remote control modes of the control device based on the way a 
user is interacting with the device. For instance, the control device can differentiate 

20 between being moved back and forth on a flat surface and being lifted up and held in the 
hand in order to adjust its functionality and transmission medium accordingly. In this 
manner, the control device is quickly and easily adapted for use in all control related 
tasks for both PC and home entertainment equipment. 
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A better understanding of the objects, advantages, features, properties and 
relationships of the subject controlling device will be obtained from the following 
detailed description and accompanying drawings which set forth an illustrative 
embodiment which is indicative of the various ways in which the principles thereof may 
5 be employed. 



BRIEF DESCRIPTION OF THE DRAWINGS 
A controlling device having integrated mouse and remote control capabilities is 
hereinafter described with reference to examples shown in the following drawings in 
10 which: 

Figure 1 illustrates an example prior art system; 

Figure 2 illustrates an exemplary control environment including an exemplary 
controlling device having integrated mouse and remote control capabilities; 

Figure 3 illustrates the various control and transmission modes of the exemplary 
15 control device of Fig. 2; 

Figure 4 illustrates a block diagram view of various components of the exemplary 
control device of Fig. 2; 

Figure 5 illustrates an exemplary method for detecting and reconfiguring the 
exemplary control device of Fig. 2 for mouse or remote control functionality; and 
20 Figures 6a and 6b illustrates an alternative embodiment of a control device having 

integrated mouse and remote control capabilities. 
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DETAILED DESCRIPTION 
Referring now to the figures, wherein like reference numerals refer to like 
elements, there is illustrated a control device 200 having integrated mouse and remote 
control capabilities. As will be appreciated from the following description, the control 
5 device 200 is generally configured to be used easily for traditional mouse-like operation, 
while remote control functionality is effected by one or more of various buttons, toggles, 
switches, touch panels, and the like configured on the control device. Additionally, one 
or more buttons on the control device may be adapted for use in both mouse and remote 
control device modes. In order to detect changes in device mode (for operation as a 
10 mouse or operation as a remote control) and also to effect mouse based operations, the 
control device includes an optical sensor, typically on the underside of control device 
200. 

The construction and operation of optical mice, i.e., those which sense movement 
by analyzing minute variations in the surface upon which they are resting, is well 

15 understood in the art. For example, "ADNS-2030 Low Power Optical Mouse Sensor 
Data Sheet," document 5988-8421EN, March 12 2003, published by Agilent 
Technologies Inc., describes the electrical and mechanical requirements and operation of 
such an optical motion sensor while "USB Wireless Optical Mouse and Multimedia 
Keyboard Solution: Designer Reference Manual," document DRM042/D Rev 0, June 

20 2003, published by Motorola Inc., describes the design of a complete wireless mouse 
system using the above referenced Agilent optical sensor. For the sake of brevity, only 
those details regarding construction and operation of the underlying optical mouse system 
which are necessary to fully elucidate the current invention will be described herein. A 
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reader desirous of additional background information may refer to the above referenced 
documents which are incorporated herein by reference in their entirety. 

With reference now to Figure 2, a control device 200 is illustrated as capable of 
commanding operation of a television 202 and a set top box 204 via wireless (e.g., IR) 
5 transmissions 206 while also able to be used as a mouse communicating with a PC 208 
via wireless (e.g., RF) transmissions 210. While the exemplary control device 200 is 
shown and described as using IR transmissions for remote control functionality and RF 
transmissions for mouse functionality, it will be understood by those skilled in the art that 
both wireless mice and remote controls are easily configured for operation using either IR 

10 and/or RF transmissions. Accordingly, the subject control device contemplates the use of 
IR and/or RF transmissions to effectuate communications. Still further, it is 
contemplated that the described controlling device may be modified to provide a single 
device that allows either or both mouse and remote control commands to be transmitted 
via wired connections to the appropriate devices to be controlled. 

15 Turning now to Figure 3, when the exemplary control device 200 is resting on a 

surface 212 such as a mouse pad, tabletop, users leg, etc., it functions as a mouse 
transmitting data 210 (e.g., representative of x-y movements 320 of the controlling device 
200, scroll wheel 307 activity, and left/right button 306 clicks) to the associated PC 208. 
However, employing methods that will be described in further detail hereafter, the 

20 functionality of the exemplary control device 100 may automatically switch from that 
associated with a mouse to that associated with a remote control when the exemplary 
control device 200 senses that it has been lifted 322 off the surface 212. In this case, 
activation of the scroll wheel 307, buttons 306, etc. results in the transmission of data 206 
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(e.g., to command appliance functions such as TV volume, mute and power, etc.) to one 
or more of the associated entertainment appliances 202, 204. Furthermore, the upper case 
of the control device 200 may also incorporate additional remote control specific buttons 
(e.g., number keys, macro keys, mode keys, etc. as described in commonly assigned U.S. 
5 Patent No. 4,959,810 which is incorporated herein by reference in its entirety) and/or a 
display (e.g., a segmented EL display panel and associated touch sensitive overlay 302 of 
the general type described in commonly assigned, co-pending U.S. Patent Application 
10/410,103 entitled "Remote Control with Local, Screen Guided, Setup" which is hereby 
incorporated by reference in its entirety). When operating in the mouse mode, the remote 

10 control specific buttons and/or EL panel 302, if any, are preferably not energized and, as 
such, the panel 302 would remain dark as illustrated in Figure 3. When operating in the 
remote control mode, however, the buttons and/or EL display panel and associated touch 
sensitive surface 302 may be activated to provide additional remote control key 
functionality as described in the above referenced '103 application. While the display 

15 302 having touch screen control keys is illustrated as being provided to an outer surface 
of the controlling device 200, it will be appreciated that a display 302 and/or further 
remote control buttons may be disposed behind a panel that is hingedly attached, for 
example, to the outer surface of the controlling device 200. 

Referring now to Fig. 4, there is illustrated in bock diagram form the hardware 

20 architecture of an exemplary control device 200. This hardware architecture may 

include, as needed for a particular application, a processor 400 coupled to one or more 
memory devices (such as a ROM memory 402, a RAM memory 404, and a non-volatile 
memory 406), a key matrix 408 (e.g., physical buttons 306, a touch sensitive panel 302.2, 
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or a combination thereof), an internal clock and timer 410, transmission circuits (e.g., JR 
412 and/or RF 414), a means to provide visual feedback to the consumer (e.g, LED 416 
and/or EL panel controller 418 and associated EL display 302.1, or the like), means to 
provide audio feedback to the user (e.g., a speaker - not illustrated), and a power supply 
5 (not illustrated). Also coupled to processor 400 may be a scroll wheel encoder 420 which 
may be of the optical or mechanical type, both well known in the art, and an optical 
mouse sensor subsystem 422. While the optical mouse sensor 422 may be strictly 
regarded as a microcontroller in its own right, since a typical sensor such as the 
exemplary Agilent ADNS-2030 includes a digital signal processor ("DSP"), memory, and 

10 self-contained programming with which to process the incoming image frames, for clarity 
in this description it will be treated simply as a component module of the system. When 
enabled by the processor 400, the optical mouse sensor subsystem 422 illuminates the 
surface 212 on which the mouse is resting via LED 424, capturing sequential images of 
surface features (frames) via lens 426 and light sensor 428 and performing a 

1 5 mathematical analysis of the differences between successive frames in order to determine 
direction and magnitude of movement, which is reported back to processor 400 for 
onward transmission to PC 208 via RF transmitter 414. As will become apparent 
hereafter, information reported back by the optical mouse sensor 422 may also be useful 
in determining whether mouse 200 is in contact with surface 212 or has been lifted away. 

20 As will be understood by those of skill in the art, the memory device(s) 402, 404 

and/or 406 include executable instructions that are intended to be executed by the 
processor 400 to control the operation of the control device 200. In this manner, the 
processor 400 may be programmed to control the various electronic components within 



7 



the control device 200, e.g., to monitor and manage the power supply to the components, 
to decode key presses and cause the transmission of command signals to entertainment 
appliances, to read and transmit mouse and scroll wheel positioning information to a PC, 
to operate user feedback means such as LEDs and/or displays, etc. In addition, the non- 
5 volatile memory 406, for example, an EEPROM or the like, may store setup data and 
parameters as necessary such that data is not required to be reloaded after battery 
changes. It is to be additionally understood that the memory devices may take the form 
of any type of readable media, such as, for example, a Smart Card, memory stick, a chip, 
a hard disk, a magnetic disk, and/or an optical disk. Still further, it will be appreciated 

10 that some or all of the illustrated memory devices may be physically incorporated within 
the same IC chip as the microprocessor 400 (a so called "microcontroller") and, as such, 
they are shown separately in Fig. 4 only for the sake of clarity. 

To cause the control device 200 to perform an action, the control device 200 is 
adapted to be responsive to events, such as a sensed consumer interaction with the key 

15 matrix 408, scroll wheel 307, etc., or detection of mouse movement by optical mouse 

sensor 422. In this connection, optical mouse sensor 422 is enabled on a periodic basis as 
further described below in conjunction with Figure 5. In response to an event, 
appropriate instructions and/or data within the memory devices are executed and/or 
accessed. For example, when a command key is activated on the control device 200 

20 while it is operating in the remote control mode (i.e., in a position lifted away from 

surface 212), the control device 200 may retrieve a code data value corresponding to the 
activated command key from a memory device and access instructions to transmit the 
retrieved code data value to a device in a format recognizable by the device. It will be 
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appreciated that the instructions within the memory devices can be used not only to cause 
the transmission of command codes and/or data to the devices but also to perform local 
operations. While not limiting, local operations that may be performed by control device 
200 when in the universal remote control mode include setting up the remote control to 
5 operate specific items of equipment (e.g., a "Sony" brand TV set or a "Philips" brand set 
top box), favorite channel setup, macro button setup, etc. Since examples of set up and 
local operations can be found in U.S. Patent Nos. 4,959,810 "Universal Remote Control 
Device," 5,255,313 "Universal Remote Control System," 5,481,256 "Favorite Key 
Command and Chained Macro Command in a Remote Control," and 6,587,067 

10 "Universal remote control with macro command capabilities" all of which are 

incorporated herein by reference in their entirety, this will not be discussed in greater 
detail herein, except to point out that one such local operation may specifically be 
provided to allow the user of the mouse/universal remote control combination to 
designate which particular remote control functions are to be made available on the 

15 mouse hard buttons and scroll wheel, for example, selecting between configurations such 
as volume up/down or channel up/down on scroll wheel, mute, power or previous 
channel on a hard button, etc. Furthermore, in this exemplary remote control 200, the EL 
panel 302.1 may be constructed as described in pending PCT patent application WO 
00/72638, which is assigned to Cambridge Consultants Ltd. and which is incorporated 

20 herein by reference in its entirety, to allow various parts of the display to be 

independently illuminated under control of the microprocessor 400 and EL display 
controller interface 418. The advantage of such a construction is that different elements 
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may be illuminated at different times, for example depending on the activity currently 
being performed by the user. 

When operating in the mouse mode (e.g., while in sensed contact with surface 
212) the control device 200 may disable the EL display 302.1 and associated portion 
5 302.2 of key matrix 408, while continuing to monitor only the portion 306 of the key 
matrix associated with the mouse buttons, as well as the input from the scroll wheel 
encoder 420 and the periodically-enabled optical mouse sensor 422. It will be 
appreciated that in cases where the scroll wheel mechanism is of the optically encoded 
type, as a power conservation measure it may also be periodically enabled in conjunction 

10 with or separately from the optical mouse sensor. Further, when operating in the mouse 
mode, various power management strategies may be practiced by the programming of 
processor 400 with regard to the periodic activation of the optical mouse sensor 
subsystem, as will be described in greater detail hereafter in conjunction with Figure 5. 

It should also be noted that while the embodiment illustrated incorporates both the 

15 wireless mouse and universal remote control transmission management functions into a 
single microcontroller 400, other implementations using separate microcontrollers for 
each operational mode are also possible. 

Turning now to Figure 5, there is illustrated in further detail the incorporation of a 
test for lift up and the automatic switching back and forth between remote control 

20 functionality and mouse functionality into an exemplary power management sequencing 
algorithm for a wireless mouse as can be found in the before mentioned Motorola 
Designer Reference Manual. In this regard, many microcontrollers designed for 
embedded applications, such as for example some members of the Motorola 68HC08 or 
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Microchip PIC families, include an ability to operate in a low power quiescent mode 
wherein all operations are suspended except for a single timer which functions to 
reawaken the main processor after a predetermined period. In this manner, a battery 
powered device such as a wireless mouse may conserve battery life by shutting itself 
5 down during periods of inactivity, while still accommodating the requirement to 
reawaken at intervals to briefly check for resumption of activity. 

When stationary and resting on surface 212, the control device is normally in state 
500. In this state, processor 400 spends most of its time in a quiescent mode as described 
above, waking at relatively long intervals and briefly enabling power to the optical mouse 

10 sensor 422 (and scroll wheel encoder 420 if required) in order to check for any activity. 
As is well known in the art, internal circuitry within the processor may also immediately 
generate a wakeup condition in response to key matrix activity, e.g., a mouse button 306 
(scroll wheel implementations and/or lift-up detectors using mechanical contacts, tilt 
switch, etc. may also be supported via this method.) If the control device 200 detects that 

1 5 it has been lifted off the surface 2 1 2, it automatically switches to remote control 

functionality, entering state 530. If, on the other hand, the control device 200 detects 
mouse activity, it enters either state 510 or 520 depending on the type, x-y motion or 
button press. In state 510, entered into upon detection of x-y movement, the optical 
mouse sensor is interrogated continuously and any motion reported to the PC. In state 

20 520, entered into as a result of button press or scroll wheel activity, the sensor is 
interrogated intermittently, but on a more frequent basis than in state 500, on the 
presumption that button or scroll activity may be a precursor to x-y motion. If mouse use 
continues, the control device 200 may transition back and forth between states 510 and 



520 as appropriate. If no activity is detected for a relatively long period of time, the 
control device will return to idle state 500. If at any time the control device 200 detects it 
has been lifted off surface 212, it automatically transitions to the remote control 
operational state 530. On entering into state 530, the control device 200 may illuminate 
5 the EL panel (if so equipped), enable buttons or touch panel keys 302.2 (it being 

understood that during mouse mode operation, only the mouse button portion 306 of the 
overall key matrix 408 may have been enabled), and enter into a low-power state waiting 
for a key press. Whenever a key is pressed (it being understood that in this context the 
term "key press" includes scroll wheel activity - e.g., the scroll wheel may be tied to 

10 volume control functionality) the control device transmits 540 the appropriate remote 
appliance control command and returns to state 530. Additionally, using the same timer 
mechanisms as described earlier, the control device 200 may periodically awaken and 
briefly transition to state 550 wherein the optical mouse sensor 422 is enabled in order to 
verify that the control device 200 is still lifted off surface 212. If it is determined that the 

15 control device 200 has been returned to the surface 212, it may automatically return to 
mouse functionality at state 520. State 550 may also incorporate an additional timing test 
(not illustrated) to shut down EL panel illumination after a certain amount of time has 
elapsed with no button activity. 

By way of further example, a test for lift up may be performed as follows: The 

20 before mentioned Agilent ADNS-2030 may be used as the optical mouse sensor 422. 

The ADNS-2030 provides a readable parameter SQUAL ("Surface QUALity") which is a 
measure of the number of distinct features visible to sensor 428 in the current field of 
view. SQUAL is reported in the form of a one-byte number between 0 and 255, and is 
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equal or close to zero if there is no surface below the sensor within the focal plane of lens 
426. In general, the focal depth of the optical systems used in mouse applications is of 
the order of a few millimeters, so it will be appreciated that whenever the mouse is lifted 
off the surface 212 on which it rests, the reported SQUAL value will rapidly reduce to 
5 near zero. By testing the SQUAL value on a regular basis, the microcontroller 400 

programming is thus able to determine if the control device 200 has been picked up from 
and/or replaced onto surface 212, and switch back and forth between remote control and 
mouse functionality accordingly. 

It will be appreciated that the above described method is not intended to be 

10 limiting. Many other detection methods are possible. For example, other brands or 
models of optical mouse sensor may use different techniques for detecting and/or 
reporting the absence of a surface below the mouse, or may not report it at all. In this 
latter case, alternative sensing means such as user activated buttons or switches, tilt based 
switches, mechanical contacts on the underside of the mouse body, gyroscopic sensors, 

15 etc., may be provided as a substitute sensing mechanism. Such alternative sensing 
mechanisms may in some cases (e.g., gyroscopic motion and direction based sensors) 
also be configured to provide the mouse and/or additional remote control capabilities. 
These alternative approaches may also be appropriate for mouse sensor implementations 
using electromechanical motion sensing methods and thus lacking inherent "lift up" 

20 detection ability. 

Figure 6 illustrates an further embodiment of a control device 200, wherein the 
control device 200 operates in a manner similar to that described above except that it 
additionally includes an IR wireless pointer subsystem constructed in accordance with the 
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teachings of U.S. Patents 5,963,145 "System for Providing Wireless Pointer Control" and 
6,271,831 "Wireless Control and Pointer System," both of like assignee and both 
incorporated herein by reference in their entirety. In general, these subsystems provide 
for user control of for example, a cursor, via the motion of a hand held pointing device 
5 transmitting an IR beacon signal 606, which signal is received and analyzed by a base 
station 600, 604 and used to determine orientation and movement of the control device 
200 relative to the base station. For the sake of brevity herein, the interested reader is 
referred to the above referenced U.S. patents for more complete details regarding the 
systemsand methods by which this is accomplished. In the instant embodiment, when 

10 the control device 200 is resting on surface 212 as illustrated in Figure 6a, it operates as a 
mouse device in the manner previously described, communicating mouse data to PC 208 
via, for example, RF transmissions 210. In this case, the receiver 600, 602 for RF signals 
210 is integral with the base station used to receive the IR beacon signals mentioned 
above, and shares a common interface 610 to PC 208. When the control device 200 is 

15 lifted from surface 212, as illustrated in Figure 6b, it automatically switches to remote 
control functionality in the manner previously described. However in this embodiment, 
immediately prior to switching into remote control mode the mouse/universal remote 
control combination transmits a final RF notification to base unit 600, which causes the 
base unit to activate its beacon receiver 604 and enable transfer of beacon-derived 

20 movement data to the PC via interface 610. As will be appreciated by those of skill in the 
art, this beacon-derived movement data may be normalized by the internal logic of the 
base unit such that the switch from optical mouse sensor data to IR beacon-derived data 
is transparent to the PC mouse decoding software, resulting in a seamless transition from 
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one method to the other Similarly, when the control device is replaced on surface 212, a 
transmission may be initiated to return the base unit 600 to mouse operation. While in 
the remote control mode, the user of such a device may thus be provided with a button 
which serves to activate the cursor control beacon, making is possible to continue to 
5 manipulate the cursor on the PC screen while also controlling an entertainment 

appliance(s) 202. While specifically described in terms of the technology disclosed in the 
above referenced U.S. patents 5,963,145 and 6,271,831, it will be appreciated that other 
free space gesture and/or pointing data generation means may also be used in a similar 
manner, for example accelerometers, tilt switches, etc. 

10 While various embodiments of a system and method for constructing a control 

device having integrated mouse and remote control functionality have been described in 
detail, it will be appreciated by those skilled in the art that various modifications and 
alternatives to those details could be developed in light of the overall teachings of the 
disclosure. Additionally, while described in the context of functional modules and 

15 illustrated using block diagram format, it is to be understood that, unless otherwise stated 
to the contrary, one or more of the described functions and/or features may be integrated 
in a single physical device and/or a software module in a software product, or one or 
more functions and/or features may be implemented in separate physical devices or 
software modules. 

20 It will also be appreciated that a detailed discussion of the actual implementation 

of each module is not necessary for an enabling understanding of the invention. Rather, 
the actual implementation of such modules would be well within the routine skill of a 
programmer and system engineer, given the disclosure herein of the system attributes, 
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functionality, and inter-relationship of the various functional modules in the system. For 
example, in applications which require only simple or no universal remote control 
functionality, the touch sensitive EL panel-based keypad portion may be omitted and 
basic remote control functionality supplied only by way of physical buttons and the scroll 
5 wheel (if provided for standard mouse operation); transmission of mouse data to a PC 
may utilize methods other than RF, e.g. IR or a hard-wired connection such as USB, 
serial RS232, or PS2; transmission of commands to consumer appliances may utilize 
methods other than IR, e.g. direct RF, wireless networking such as IEEE 802.1 1 or 
Bluetooth, acoustic; etc. Furthermore, the consumer appliances controlled may not be 

10 limited to entertainment devices but may also include thermostats, lighting, drapes, 

domestic appliances, alarms, irrigation systems, and the like, as well as communication 
systems, home gateways, and other computing and home automation devices which may 
become part of or associated with the user's overall media center home configuration. 
Therefore, a person skilled in the art, applying ordinary skill, will be able to practice the 

15 invention set forth in the claims without undue experimentation. It will be additionally 
appreciated that the particular concepts disclosed are meant to be illustrative only and not 
limiting as to the scope of the invention which is to be given the full breadth of the 
appended claims and any equivalents thereof. 

All documents cited within this application for patent are hereby incorporated by 

20 reference in their entirety. 
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